Intelligent systems have traditionally been designed as tools rather than collaborators, often lacking critical characteristics that collaboration partnerships require. Recent advances in large language model (LLM) agents open new opportunities for human-LLM-agent collaboration by enabling natural communication and various social and cognitive behaviors. Yet it remains unclear whether principles of computer-mediated collaboration established in HCI and CSCW persist, change, or fail when humans collaborate with LLM agents. To support systematic investigations of these questions, we introduce an open and configurable research platform for HCI researchers. The platform's modular design allows seamless adaptation of classic CSCW experiments and manipulation of theory-grounded interaction controls. We demonstrate the platform's effectiveness and usability through two case studies: (1) re-implementing the classic human-human-collaboration task Shape Factory as a between-subject human-agent-collaboration experiment with 16 participants, and (2) a participatory cognitive walkthrough with five HCI researchers to refine workflows and interfaces for experiment setup and analysis.

Gap the (Theory of) Mind: Sharing Beliefs About Teammates' Goals Boosts Collaboration Perception, Not Performance Abstract --In human-agent teams, openly sharing goals is often assumed to enhance planning, collaboration, and effectiveness. However, direct communication of these goals is not always feasible, requiring teammates to infer their partner's intentions through actions. Building on this, we investigate whether an AI agent's ability to share its inferred understanding of a human teammate's goals can improve task performance and perceived collaboration. Through an experiment comparing three conditions--no recognition (NR), viable goals (VG), and viable goals on-demand (VGod)--we find that while goal-sharing information did not yield significant improvements in task performance or overall satisfaction scores, thematic analysis suggests that it supported strategic adaptations and subjective perceptions of collaboration. Cognitive load assessments revealed no additional burden across conditions, highlighting the challenge of balancing informativeness and simplicity in human-agent interactions. These findings highlight the nuanced trade-off of goal-sharing: while it fosters trust and enhances perceived collaboration, it can occasionally hinder objective performance gains. In human-agent collaboration, effective teamwork often depends on the agent's ability to interpret and act upon the human teammate's intentions. Ad-hoc teamwork [1], where team members must collaborate effectively without prior planning, exemplifies contexts where this capability is critical. Explainable AI (XAI) aims to address this by enhancing transparency and interpretability in AI systems, fostering shared mental models, trust, and mutual understanding [2], [3].

Recent advancements in language models (LMs) have sparked growing interest in developing LM agents. While fully autonomous agents could excel in many scenarios, numerous use cases inherently require them to collaborate with humans due to humans' latent preferences, domain expertise, or need for control. To facilitate the study of human-agent collaboration, we present Collaborative Gym (Co-Gym), a general framework enabling asynchronous, tripartite interaction among agents, humans, and task environments. We instantiate Co-Gym with three representative tasks in both simulated and real-world conditions, and propose an evaluation framework that assesses both the collaboration outcomes and processes. Our findings reveal that collaborative agents consistently outperform their fully autonomous counterparts in task performance within those delivered cases, achieving win rates of 86% in Travel Planning, 74% in Tabular Analysis, and 66% in Related Work when evaluated by real users. However, our study also highlights significant challenges in developing collaborative agents, requiring advancements in core aspects of intelligence -- communication capabilities, situational awareness, and balancing autonomy and human control.

In recent developments within the research community, the integration of Large Language Models (LLMs) in creating fully autonomous agents has garnered significant interest. Despite this, LLM-based agents frequently demonstrate notable shortcomings in adjusting to dynamic environments and fully grasping human needs. In this work, we introduce the problem of LLM-based human-agent collaboration for complex task-solving, exploring their synergistic potential. In addition, we propose a Reinforcement Learning-based Human-Agent Collaboration method, ReHAC. This approach includes a policy model designed to determine the most opportune stages for human intervention within the task-solving process. We construct a human-agent collaboration dataset to train this policy model in an offline reinforcement learning environment. Our validation tests confirm the model's effectiveness. The results demonstrate that the synergistic efforts of humans and LLM-based agents significantly improve performance in complex tasks, primarily through well-planned, limited human intervention. Datasets and code are available at: https://github.com/XueyangFeng/ReHAC.

Empirical studies with humans and agents demonstrate that the nature and forms of information required by the human differ depending on the design of the relationship between the participants — a relationship that is sometimes characterised using the concept of levels of autonomy, though the usefulness of that characterisation has recently been questioned. Therefore, understanding how people work with automation and how to design automated systems to better support people, is a field long studied, but of growing importance. Our current work seeks to contribute to the design of representations and algorithms that can be deployed in such contexts.

Virtual and robotic agents must be able to understand "communicative acts" (utterances, gestures, controlled facial expressions etc.) if they are to interact and collaborate with humans. For researchers in AI, HCI, HRI and related fields, automatic comprehension of communicative acts has turned out to be a very tough nut to crack. Drawing on recent research from cognitive science and evolutionary psychology, the paper argues that an insufficient conceptualization of "context" is at the heart of this problem, and that we should focus on very simple, non-linguistic communicative acts (pointing gestures etc.) in order to investigate how agents can comprehend communicative acts in realistic contexts. I propose a tripartite model of context which is informed by experimental research on how humans recognize objects (via "affordances"), causal relations among objects, and the collaborative activities of fellow-humans. The model is not a formal one, but detailed enough to help in the development of comprehension algorithms in future research.